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DETAILED ACTION 
Response to Amendment 

1 . Amendment A was received on May 24, 2004 has been entered and made 
of record. Currently, claims 1, 3-16,18-25, 27-40, 42-48 are pending. 

Drawings 

2. The drawings of figures 1 ,5A, 6A and 6F were received on May 24, 2004. 
These drawings are acceptable. 

Specification 

3. Due to the amendment, the objection to the specification has been 
withdrawn. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 

U.S.C. 102 that form the basis for the rejections under this section made in this 

Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in 
public use or on sale in this country, more than one year prior to the date of application for patent in 
the United States. 

5. Claims 1,3-9,13,18-25,27-33,37,42-48 are rejected under 35 

U.S.C. 102(b) as being anticipated by Katayama et al. (US Patent 6,704,041 B1). 
Regarding claim 1 , Katayama et al. discloses a method comprising: 
a) receiving a reference image (IMAGE PICKUP AREA 1 of fig. 3) and 
a first image (IMAGE PICKUP AREA 2 of fig. 3), the first image (IMAGE PICKUP 
AREA 2 of fig. 3) having a perimeter having a first shape (rectangular perimeter); 
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b) determining a relative position (POINT P1 L of fig. 5) of the first 
image (IMAGE PICKUP AREA 2 of fig. 3) and the reference image (IMAGE 
PICKUP AREA 1 of fig. 3) 

POINT P1L is used to match both images using a search area 202 of fig. 
5; 

c) using the first image (IMAGE PICKUP AREA 2 of fig. 3), the 
reference image (IMAGE PICKUP AREA 1 of fig. 3), and the relative position 
(POINT P1L of fig. 5) to generate a first corrected image (The image on the right 
side of Fig. 8B is generated from an "IMAGE COMBINATION PROCESSING 
UNIT" of fig. 1.) having less perspective distortion (The image on the right side of 
Fig. 8B is "a high-quality combined image (Katayama et al., col. 5, lines 63,64).") 
relative to the reference image (IMAGE PICKUP AREA 1 of fig. 3) than the first 
image (IMAGE PICKUP AREA 2 of fig. 3) has, the first corrected image (IMAGE 
PICKUP AREA 2 of fig. 3) having a perimeter having a first corrected shape 
different from the first shape (Trapezoidal shape on the right side of fig. 8B.); 

d) using the first corrected shape (The trapezoidal shape corresponds 
to an output of step "S43" of fig. 16) to determine a focal length (Fig. 16, num. 
S44 is a focal length estimation for the focal length "f .) and rotation angles (Fig. 
18, co n-i to <x>3 ) of a camera (Fig. 3 and Fig. 18 has a camera located at the 
central point or the radial lines.) associated with the first image (A rotation angle 
is determined for each image of the camera as mentioned in col. 10, lines 22,23.) 

The focal length and rotation angles are determined using equations 13 
and 14 of col. 10, lines 26 and 36 during step "S44"; and 
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e) projecting the first image (IMAGE PICKUP AREA 2 of fig. 3) on a 
surface ("cylindrical mapping conversion" at step "S46" of fig. 16 and mentioned 
in col. 11, line 13 and shown in fig. 23.) based on the focal length (fig. 14, num. 
112) and rotation angles (fig. 18, a> n-i to co 3 ) of the camera (Fig. 18 has a 
camera located at the central point or radial lines) associated with the first image 
(IMAGE PICKUP AREA 2 of fig. 3 has a corresponding angle that was 
determined using the method of fig. 18.) 

Using fig. 16, the focal length and rotation angles from step "S44" are 
inputted to step "S46", which generated a cylindrical image, via step "S45". 

Regarding claim 3, Katayama et al. discloses the method of claim 1 , 
further comprising: 

projecting the reference image (IMAGE PICKUP AREA 1 of fig. 3) on the 
surface ("cylindrical mapping conversion" at step "S46" of fig. 16 and mentioned 
in col. 1 1 , line 1 3 and shown in fig. 23.). 

Note that the reference image also corresponds with the image on the left 
of fig. 8A that is combined with the first image on the right in fig. 8A and projected 
on a cylindrical surface. 

Regarding claim 4, Katayama et al. discloses the method of claim 3, 
further comprising: 

merging the projected reference image (IMAGE PICKUP AREA 1 of fig. 3) 
and the projected first image (IMAGE PICKUP AREA 2 of fig. 3) to form a 
panoramic image (fig. 24, "COMBINED IMAGE"). 
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Both images are merged using a horizontal line that runs through each 
image as shown in fig. 23 to form the panoramic image of fig. 24. 

Regarding claim 5, Katayama et al. discloses the method of claim 3 further 
comprising: 

projecting a three-dimensional object (The image on the right of figure 8B 
is a three dimensional object as shown by the length, width, and depth of the 
image) onto the surface ("cylindrical mapping conversion" at step "S46" of fig. 16 
and mentioned in col. 11, line 13 and shown in fig. 23.). ; 

merging the projected three-dimensional object (The image on the right of 
figure 8B.), the reference image (IMAGE PICKUP AREA 1 of fig. 3) and the first 
image (IMAGE PICKUP AREA 2 of fig. 3) to form a panoramic image (fig. 24, 
"COMBINED IMAGE"). 

Note that the three-dimensional object corresponds to a third image, 
"IMAGE PICKUP AREA 3" which is merged with the other images to form a 
panoramic image of fig. 24. 

Regarding claims 6,7, and 8, Katayama discloses the method of claim 1 
wherein the surface is cylindrical (fig. 24), spherical ("spherical" in col. 8, line 23) 
and planar (fig. 22). 

Claim 9 is similar to claim 1 except for requiring another image (IMAGE 
PICKUP AREA 3 of fig. 3) that is processed in the same way of claim 1 . 



Application/Control Timber: 09/848,01 7 W Page 6 

Art Unit: 2621 

Regarding claim 13, Katayama discloses the method of claim 1, wherein 
the rotation angles (fig. 18, co N -i to co 3 ) of the camera (Fig. 18 has a camera 
located at the central point or radial lines) associated with the first image 
(IMAGE PICKUP AREA 2 of fig. 3 has a corresponding angle that was 
determined using the method of fig. 18.) is measured (A displacement, of fig. 
18, is measured as indicated by an arrow beneath xi that corresponds to the 
rotation angles, co n-i to 03.) relative to rotation angles (The rotation angles, co n-i 
to co 3 , are equally spaced from each other to form a total angle of 360 degrees as 
mentioned in col. 10, lines 32,33.) of a camera (The camera of fig. 18.) 
associated with the reference image (IMAGE PICKUP AREA 1 of fig. 3 
corresponds with an image of fig. 18 for determining the rotation angles, co N -i to 
co 3 .). 

Claim 18 has different claim language than claim 1, but both claims are 
directed to the same function. Thus, claim 18 has been addressed in claim 1. 

Regarding claim 19, Katayama et al. discloses the method of claim 18, 
further comprising: 

determining a position offset ("D" of fig. 22) of the first segment (IMAGE 2 
of fig. 22) of the view ("H" of fig. 22 is a view of a whole scene.) relative to the 
reference segment (IMAGE x of fig. 22) of the view, wherein correcting for 
perspective distortion (Fig. 23 shows all images aligned properly) is based on the 
determined position offset (The position offset is eliminated from fig. 22 to 
generate the image of fig. 23.). 
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Regarding claim 20, Katayama et al. discloses the method of claim 18 
wherein the perimeter of the first image (IMAGE PICKUP AREA 2 of fig. 3 has a 
rectangular perimeter.) includes at least a first reference point and a second 
reference point (The image on the right of fig. 6B corresponds with the IMAGE 
PICKUP AREA 2 of fig. 3 has multiple reference points, V.) and correcting for 
perspective distortion (The image on the right side of Fig. 8B is "a high-quality 
combined image (Katayama et al., col. 5, lines 63,64).") alters the shape of the 
perimeter (The image on the right of fig. 8B has a trapezoidal shape that 
corresponds with the image on the right of fig. 6B. Thus, the perimeter of the first 
image changes from a rectangular shape to a trapezoidal shape.) of the first 
image (IMAGE PICKUP AREA 2 of fig. 3 has a rectangular perimeter.) by moving 
the first reference point relative to the second reference point (The multiple points 
of fig. 6B are "moved" as mentioned in col. 3, line 65). 

Regarding claim 21 , Katayama et al. discloses the method of claim 20 
wherein the first and second reference points (The image on the right of fig. 6B 
corresponds with the IMAGE PICKUP AREA 2 of fig. 3 has multiple reference 
points, V.) are vertices (The point V of figure 6B are corner points of the 
multiple V points.) defined by the shape (The reference points are arranged 
within the perimeter of the image on the right of fig. 6B.) of the perimeter 
(rectangular perimeter) of the first image (IMAGE PICKUP AREA 2 of fig. 3 that 
corresponds to the image on the right of fig. 6B). 
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Regarding claim 22, Katayama et al. discloses the method of claim 21 
wherein the shape of the perimeter of the first image is rectangular (addressed in 
claim 21 ) and correcting for perspective distortion (The image on the right side of 
Fig. 8B is "a high-quality combined image (Katayama et al., col. 5, lines 63,64).") 
alters the shape of the perimeter of the first image into a trapezoid (The image's 
shape on the right of fig. 6B is changed to a trapezoid image as shown on the 
right of fig. 8B.). 

Regarding claim 23, Katayama et al. discloses the method of claim 1 , 
wherein determining the focal length (Fig. 16, num. S44 is a focal length 
estimation.) and rotation angles (Fig. 18, co N -i to co 3 ) is further based on the 
shape of the perimeter of the first image (The output of step "S43" is the image of 
fig. 8B that is inputted to the next step "S44".). 

Regarding claim 24, Katayama et al. discloses the method of claim 1 
wherein the perimeter (rectangular) of the first image (IMAGE PICKUP AREA 2 
of fig. 3) has the same shape as the perimeter (rectangular) of the reference 
image (IMAGE PICKUP AREA 1 of fig. 3). Both images have a rectangular 
perimeter. 

Claims 25,27-33,37,42-48 are respectively addressed in claims 1 ,3- 
9,13,18-24 except for the limitation of an article comprising a machine-readable 
medium on which are tangibly stored machine-executable instructions which is 
disclosed by Katayama in col. 11, lines 38-40 as a "program stored on a 
recording medium". 
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Allowable Subject Matter 

6. Claims 10 and 34 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all 
of the limitations of the base claim and any intervening claims. 

Claim 10 and 34 are allowable because the prior art does not teach or 
suggest the step of "comparing the estimated shape and the actual shape of the 
perimeter of the corrected version of the first image". 

The closest prior art, Katayama et al. (US Patent 6,704,041 B2), teaches 
comparing (Fig. 12, "S23") an image's data points (Fig. 12, "S22") with a 
predetermined threshold Th\ 

The benefit of claim 10 allows alteration in the perimeter of the image to 
compute the rotation angle and focal length, thus the computing resources and 
computing time is reduced. 

Conclusion 

7. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Katayama et al. (US Patent 6,389,179 B1) is pertinent as teaching a 
method of merging two images (121 and I23) to a reference image (I22) with 
perspective distortion (L21 and L23) as shown in figures 7 and 8. 
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8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1 .136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

9. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Dennis Rosario-Vasquez whose telephone 
number is 703-305-5431. The examiner can normally be reached on 9-5. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Leo Boudreau can be reached on 703-305-4706. The fax 
phone number for the organization where this application or proceeding is 
assigned is 703-872-9306. 





Application/Control IWhber: 09/848,017 W I 

Art Unit: 2621 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 
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